Dario Amodei — Machines of Loving Grace

#clippings

How AI Could Transform the World for the Better
人工智能如何让世界变得更美好

I think and talk a lot about the risks of powerful AI. The company I’m the CEO of, Anthropic, does a lot of research on how to reduce these risks. Because of this, people sometimes draw the conclusion that I’m a pessimist or “doomer” who thinks AI will be mostly bad or dangerous. I don’t think that at all. In fact, one of my main reasons for focusing on risks is that they’re the only thing standing between us and what I see as a fundamentally positive future. I think that most people are underestimating just how radical the upside of AI could be, just as I think most people are underestimating how bad the risks could be.
我经常思考和讨论强大 AI 带来的风险。作为 Anthropic 公司的 CEO，我们投入大量研究来降低这些风险。正因如此，人们有时会认为我是个悲观主义者或"末日论者"，觉得我认为 AI 主要会带来负面影响或危险。我完全不这么认为。事实上，我专注于风险研究的主要原因在于，这些风险是我们与我所预见的基本上积极的未来之间唯一的障碍。我认为大多数人低估了 AI 可能带来的根本性积极变革，正如他们也低估了潜在风险可能有多严重。

In this essay I try to sketch out what that upside might look like—what a world with powerful AI might look like if everything goes right. Of course no one can know the future with any certainty or precision, and the effects of powerful AI are likely to be even more unpredictable than past technological changes, so all of this is unavoidably going to consist of guesses. But I am aiming for at least educated and useful guesses, which capture the flavor of what will happen even if most details end up being wrong. I’m including lots of details mainly because I think a concrete vision does more to advance discussion than a highly hedged and abstract one.
在这篇文章中，我试图勾勒出这种积极前景可能呈现的模样——如果一切顺利，一个拥有强大人工智能的世界会是什么样子。当然，没有人能确切或精准地预知未来，而强大 AI 带来的影响可能比过去的技术变革更加难以预测，因此这一切不可避免地只能由猜测构成。但我的目标是至少做出有根据且有用的猜测，即便大多数细节最终被证明是错误的，也能捕捉到未来发展的基调。我之所以包含大量细节，主要是因为我认具体化的愿景比高度保留且抽象的论述更能推动讨论。

First, however, I wanted to briefly explain why I and Anthropic haven’t talked that much about powerful AI’s upsides, and why we’ll probably continue, overall, to talk a lot about risks. In particular, I’ve made this choice out of a desire to:
不过首先，我想简要解释为什么我和 Anthropic 之前较少谈及强大 AI 的积极面，以及为什么总体上我们可能仍会继续重点讨论风险。特别是，我做出这一选择是出于以下考虑：

Yet despite all of the concerns above, I really do think it’s important to discuss what a good world with powerful AI could look like, while doing our best to avoid the above pitfalls. In fact I think it is critical to have a genuinely inspiring vision of the future, and not just a plan to fight fires. Many of the implications of powerful AI are adversarial or dangerous, but at the end of it all, there has to be something we’re fighting for, some positive-sum outcome where everyone is better off, something to rally people to rise above their squabbles and confront the challenges ahead. Fear is one kind of motivator, but it’s not enough: we need hope as well.
然而，尽管存在上述所有担忧，我确实认为讨论强大 AI 可能带来的美好世界至关重要，同时尽力避免上述陷阱。事实上，我认为拥有一个真正鼓舞人心的未来愿景而不仅仅是灭火计划是至关重要的。强大 AI 的许多影响是对抗性或危险的，但归根结底，我们必须为之奋斗的是某种正和结果，让每个人都变得更好，某种激励人们超越琐碎争执、共同应对未来挑战的东西。恐惧是一种动力，但还不够：我们同样需要希望。

The list of positive applications of powerful AI is extremely long (and includes robotics, manufacturing, energy, and much more), but I’m going to focus on a small number of areas that seem to me to have the greatest potential to directly improve the quality of human life. The five categories I am most excited about are:
强大 AI 的积极应用清单非常长（包括机器人技术、制造业、能源等众多领域），但我将聚焦于少数几个在我看来最有可能直接提升人类生活质量的领域。我最期待的五个类别是：

  1. Biology and physical health
    生物学与身体健康
  2. Neuroscience and mental health
    神经科学与心理健康
  3. Economic development and poverty
    经济发展与贫困
  4. Peace and governance 和平与治理
  5. Work and meaning 工作与意义

My predictions are going to be radical as judged by most standards (other than sci-fi “singularity” visions), but I mean them earnestly and sincerely. Everything I’m saying could very easily be wrong (to repeat my point from above), but I’ve at least attempted to ground my views in a semi-analytical assessment of how much progress in various fields might speed up and what that might mean in practice. I am fortunate to have professional experience in both biology and neuroscience, and I am an informed amateur in the field of economic development, but I am sure I will get plenty of things wrong. One thing writing this essay has made me realize is that it would be valuable to bring together a group of domain experts (in biology, economics, international relations, and other areas) to write a much better and more informed version of what I’ve produced here. It’s probably best to view my efforts here as a starting prompt for that group.
我的预测按大多数标准衡量会显得激进（除了科幻式的“奇点”愿景 ），但我是真诚且认真地提出这些观点。我所说的一切都很可能出错（重申前文的观点），但至少我尝试基于对各领域进展可能加速程度及其实际影响的半分析性评估来形成这些看法。我幸运地拥有生物学和神经科学领域的专业经验，并在经济发展方面是个有见识的业余爱好者，但我确信会犯不少错误。撰写本文让我意识到，召集一个由各领域专家（生物学、经济学、国际关系等）组成的团队来撰写一个比我这里所呈现的更优质、信息更充分的版本将极具价值。或许最好将我的努力视为对该团队的一个启动提示。

Basic assumptions and framework基本假设与框架

To make this whole essay more precise and grounded, it’s helpful to specify clearly what we mean by powerful AI (i.e. the threshold at which the 5-10 year clock starts counting), as well as laying out a framework for thinking about the effects of such AI once it’s present.
为了使整篇文章更加精确和扎实，明确界定我们所说的强大 AI（即 5-10 年倒计时开始的阈值）是有帮助的，同时还需要建立一个框架来思考这种 AI 一旦出现后可能产生的影响。

What powerful AI (I dislike the term AGI) will look like, and when (or if) it will arrive, is a huge topic in itself. It’s one I’ve discussed publicly and could write a completely separate essay on (I probably will at some point). Obviously, many people are skeptical that powerful AI will be built soon and some are skeptical that it will ever be built at all. I think it could come as early as 2026, though there are also ways it could take much longer. But for the purposes of this essay, I’d like to put these issues aside, assume it will come reasonably soon, and focus on what happens in the 5-10 years after that. I also want to assume a definition of what such a system will look like, what its capabilities are and how it interacts, even though there is room for disagreement on this.
强大 AI（我不喜欢 AGI 这个术语） 会是什么样子，以及它何时（或是否）会出现，本身就是一个庞大的话题。我曾公开讨论过这个问题，甚至可以单独写一篇文章来阐述（未来我可能会这么做）。显然，许多人对强大 AI 即将问世持怀疑态度，有些人甚至怀疑它是否会被造出来。我认为它最早可能在 2026 年出现，但也有可能需要更长时间。但为了本文的讨论，我想暂时搁置这些问题，假设它会在不久的将来出现，并重点关注此后 5-10 年内会发生什么。我还想假设这样一个系统的形态、能力以及交互方式，尽管在这方面存在争议空间。

By powerful AI, I have in mind an AI model—likely similar to today’s LLM’s in form, though it might be based on a different architecture, might involve several interacting models, and might be trained differently—with the following properties:
我心目中的强大 AI，是指一种 AI 模型——可能在形式上类似于今天的LLM，尽管它可能基于不同的架构，可能涉及多个相互作用的模型，并且可能采用不同的训练方式——具有以下特性：

We could summarize this as a “country of geniuses in a datacenter”.
我们可以将其概括为“数据中心里的天才国度”。

Clearly such an entity would be capable of solving very difficult problems, very fast, but it is not trivial to figure out how fast. Two “extreme” positions both seem false to me. First, you might think that the world would be instantly transformed on the scale of seconds or days (“ the Singularity ”), as superior intelligence builds on itself and solves every possible scientific, engineering, and operational task almost immediately. The problem with this is that there are real physical and practical limits, for example around building hardware or conducting biological experiments. Even a new country of geniuses would hit up against these limits. Intelligence may be very powerful, but it isn’t magic fairy dust.
显然，这样的实体能够以极快的速度解决非常困难的问题，但要准确衡量其速度并非易事。在我看来，两种“极端”立场都站不住脚。首先，有人可能认为世界会在几秒或几天内发生翻天覆地的变化（即“奇点”），因为超级智能会自我迭代，几乎立即解决所有可能的科学、工程和操作任务。问题在于现实中存在物理和实际限制，比如硬件制造或生物实验的耗时。即使是一个由天才组成的新国度也会遭遇这些瓶颈。智能或许无比强大，但它并非魔法仙尘。

Second, and conversely, you might believe that technological progress is saturated or rate-limited by real world data or by social factors, and that better-than-human intelligence will add very little. This seems equally implausible to me—I can think of hundreds of scientific or even social problems where a large group of really smart people would drastically speed up progress, especially if they aren’t limited to analysis and can make things happen in the real world (which our postulated country of geniuses can, including by directing or assisting teams of humans).
其次，反过来，你可能认为技术进步已经饱和，或者受到现实世界数据或社会因素的限制，而超越人类智能的进步将微乎其微 。在我看来，这同样难以置信——我能想到数百个科学甚至社会问题，如果有一群真正聪明的人参与其中，将极大地加速进展，特别是如果他们不仅限于分析，还能在现实世界中促成改变（我们假设的天才国度可以做到这一点，包括指挥或协助人类团队）。

I think the truth is likely to be some messy admixture of these two extreme pictures, something that varies by task and field and is very subtle in its details. I believe we need new frameworks to think about these details in a productive way.
我认为真相很可能是这两种极端图景的某种混乱混合体，具体细节会因任务和领域而异且极为微妙。我们需要新的框架来以富有成效的方式思考这些细节。

Economists often talk about “factors of production”: things like labor, land, and capital. The phrase “marginal returns to labor/land/capital” captures the idea that in a given situation, a given factor may or may not be the limiting one – for example, an air force needs both planes and pilots, and hiring more pilots doesn’t help much if you’re out of planes. I believe that in the AI age, we should be talking about the marginal returns to intelligence, and trying to figure out what the other factors are that are complementary to intelligence and that become limiting factors when intelligence is very high. We are not used to thinking in this way—to asking “how much does being smarter help with this task, and on what timescale?”—but it seems like the right way to conceptualize a world with very powerful AI.
经济学家经常谈论“生产要素”：如劳动力、土地和资本。短语“劳动力/土地/资本的边际回报”体现了在特定情境下，某一要素可能是也可能不是限制因素——例如，空军既需要飞机也需要飞行员，如果没有飞机，雇佣更多飞行员也无济于事。我认为在 AI 时代，我们应该讨论智力的边际回报 ，并试图找出与智力互补的其他要素，这些要素在智力极高时会成为限制因素。我们不习惯这样思考——去问“在这项任务中变得更聪明有多大帮助，以及在什么时间尺度上？”——但这似乎是理解拥有非常强大 AI 的世界的正确方式。

My guess at a list of factors that limit or are complementary to intelligence includes:
我猜测限制或补充智能的因素列表包括：

There is a further distinction based on timescales. Things that are hard constraints in the short run may become more malleable to intelligence in the long run. For example, intelligence might be used to develop a new experimental paradigm that allows us to learn in vitro what used to require live animal experiments, or to build the tools needed to collect new data (e.g. the bigger particle accelerator), or to (within ethical limits) find ways around human-based constraints (e.g. helping to improve the clinical trial system, helping to create new jurisdictions where clinical trials have less bureaucracy, or improving the science itself to make human clinical trials less necessary or cheaper).
基于时间尺度还有进一步的区分。短期内难以突破的硬性约束，长期来看可能会因智能而变得更具可塑性。例如，利用智能可以开发新的实验范式，让我们通过体外实验获取过去必须依赖活体动物实验才能得到的数据；或是建造收集新数据所需的工具（比如更大型的粒子加速器）；又或是在伦理允许范围内，设法绕过以人类为基础的限制（比如协助改进临床试验体系，帮助建立临床试验官僚程序更少的新司法管辖区，或是通过改进科学方法本身来减少对人类临床试验的依赖或降低其成本）。

Thus, we should imagine a picture where intelligence is initially heavily bottlenecked by the other factors of production, but over time intelligence itself increasingly routes around the other factors, even if they never fully dissolve (and some things like physical laws are absolute). The key question is how fast it all happens and in what order.
因此，我们应该想象这样一幅图景：起初，智能受到其他生产要素的严重制约，但随着时间的推移，智能本身会越来越多地绕过其他因素，即使这些因素永远不会完全消失（而像物理定律这样的东西是绝对的） 。关键问题在于这一切发生的速度有多快，以及顺序如何。

With the above framework in mind, I’ll try to answer that question for the five areas mentioned in the introduction.
基于上述框架，我将尝试针对引言中提到的五个领域来回答这个问题。

1. Biology and health 1. 生物学与健康

Biology is probably the area where scientific progress has the greatest potential to directly and unambiguously improve the quality of human life. In the last century some of the most ancient human afflictions (such as smallpox) have finally been vanquished, but many more still remain, and defeating them would be an enormous humanitarian accomplishment. Beyond even curing disease, biological science can in principle improve the baseline quality of human health, by extending the healthy human lifespan, increasing control and freedom over our own biological processes, and addressing everyday problems that we currently think of as immutable parts of the human condition.
生物学可能是科学进步最有可能直接且明确地提升人类生活质量的领域。上个世纪，一些最古老的人类疾病（如天花）终于被征服，但仍有更多疾病存在，战胜它们将是巨大的人道主义成就。除了治愈疾病，生物科学原则上还能通过延长人类健康寿命、增强对我们自身生物过程的控制和自由，以及解决我们目前认为不可改变的人类生存日常问题，来提升人类健康的基础水平。

In the “limiting factors” language of the previous section, the main challenges with directly applying intelligence to biology are data, the speed of the physical world, and intrinsic complexity (in fact, all three are related to each other). Human constraints also play a role at a later stage, when clinical trials are involved. Let’s take these one by one.
在前文提到的“限制因素”语境下，将智能技术直接应用于生物学领域的主要挑战在于数据获取、物理世界运行速度以及内在复杂性（实际上这三者相互关联）。当涉及临床试验阶段时，人类自身的局限性也会产生影响。让我们逐一剖析这些因素。

Experiments on cells, animals, and even chemical processes are limited by the speed of the physical world: many biological protocols involve culturing bacteria or other cells, or simply waiting for chemical reactions to occur, and this can sometimes take days or even weeks, with no obvious way to speed it up. Animal experiments can take months (or more) and human experiments often take years (or even decades for long-term outcome studies). Somewhat related to this, data is often lacking—not so much in quantity, but quality: there is always a dearth of clear, unambiguous data that isolates a biological effect of interest from the other 10,000 confounding things that are going on, or that intervenes causally in a given process, or that directly measures some effect (as opposed to inferring its consequences in some indirect or noisy way). Even massive, quantitative molecular data, like the proteomics data that I collected while working on mass spectrometry techniques, is noisy and misses a lot (which types of cells were these proteins in? Which part of the cell? At what phase in the cell cycle?).
对细胞、动物甚至化学过程的实验都受限于物理世界的速度：许多生物实验方案涉及培养细菌或其他细胞，或者仅仅是等待化学反应发生，这有时可能需要数天甚至数周时间，且没有明显的方法可以加速。动物实验可能需要数月（或更长时间），而人体实验通常需要数年（对于长期结果研究甚至可能需要数十年）。与此相关的是，数据往往不足——不是数量上的不足，而是质量上的不足：总是缺乏清晰、明确的数据，这些数据能将感兴趣的生物效应与其他成千上万的混杂因素区分开来，或者能在特定过程中进行因果干预，或者能直接测量某些效应（而不是通过某种间接或有噪声的方式推断其结果）。即使是海量的定量分子数据，比如我在质谱技术工作中收集的蛋白质组学数据，也存在噪声且遗漏了大量信息（这些蛋白质存在于哪些类型的细胞中？细胞的哪个部分？处于细胞周期的哪个阶段？）。

In part responsible for these problems with data is intrinsic complexity: if you’ve ever seen a diagram showing the biochemistry of human metabolism, you’ll know that it’s very hard to isolate the effect of any part of this complex system, and even harder to intervene on the system in a precise or predictable way. And finally, beyond just the intrinsic time that it takes to run an experiment on humans, actual clinical trials involve a lot of bureaucracy and regulatory requirements that (in the opinion of many people, including me) add unnecessary additional time and delay progress.
部分导致这些数据问题的原因在于内在复杂性：如果你曾见过展示人体新陈代谢生物化学过程的图表，就会明白很难孤立分析这个复杂系统中任何部分的影响，更难以精确或可预测的方式干预该系统。最后，除了在人体上进行实验所需的内在时间外，实际的临床试验还涉及大量官僚程序和监管要求（包括我在内的许多人认为），这些额外因素不必要地延长了时间并延缓了进展。

Given all this, many biologists have long been skeptical of the value of AI and “big data” more generally in biology. Historically, mathematicians, computer scientists, and physicists who have applied their skills to biology over the last 30 years have been quite successful, but have not had the truly transformative impact initially hoped for. Some of the skepticism has been reduced by major and revolutionary breakthroughs like AlphaFold (which has just deservedly won its creators the Nobel Prize in Chemistry) and AlphaProteo, but there’s still a perception that AI is (and will continue to be) useful in only a limited set of circumstances. A common formulation is “AI can do a better job analyzing your data, but it can’t produce more data or improve the quality of the data. Garbage in, garbage out”.
鉴于这一切，许多生物学家长期以来对人工智能乃至更广义的“大数据”在生物学中的价值持怀疑态度。过去 30 年间，将数学、计算机科学和物理学技能应用于生物学的学者虽取得了显著成就，但并未实现最初期待的颠覆性变革。尽管 AlphaFold（其创造者刚实至名归地斩获诺贝尔化学奖）和 AlphaProteo 等重大突破性成果减轻了部分质疑，但人们仍普遍认为人工智能的应用范围（并将持续）局限于特定场景。一个典型观点是：“AI 能更高效地分析数据，但无法创造新数据或提升数据质量——输入垃圾，输出仍是垃圾”。

But I think that pessimistic perspective is thinking about AI in the wrong way. If our core hypothesis about AI progress is correct, then the right way to think of AI is not as a method of data analysis, but as a virtual biologist who performs all the tasks biologists do, including designing and running experiments in the real world (by controlling lab robots or simply telling humans which experiments to run – as a Principal Investigator would to their graduate students), inventing new biological methods or measurement techniques, and so on. It is by speeding up the whole research process that AI can truly accelerate biology. I want to repeat this because it’s the most common misconception that comes up when I talk about AI’s ability to transform biology: I am not talking about AI as merely a tool to analyze data. In line with the definition of powerful AI at the beginning of this essay, I’m talking about using AI to perform, direct, and improve upon nearly everything biologists do.
但我认为这种悲观视角是以错误的方式看待 AI。如果我们关于 AI 进步的核心假设是正确的，那么正确理解 AI 的方式不应将其视为数据分析工具，而应视作一名虚拟生物学家——它能完成生物学家所有的工作，包括在现实世界中设计和运行实验（通过操控实验室机器人或直接指导人类进行实验，就像首席研究员对研究生所做的那样）、发明新的生物研究方法或测量技术等等。正是通过加速整个研究流程，AI 才能真正推动生物学发展。我要重申这一点，因为当我谈及 AI 变革生物学的能力时，这是最常见的误解：我所说的 AI 不仅仅是数据分析工具。正如本文开头对强大 AI 的定义，我指的是利用 AI 来执行、指导并改进生物学家几乎所有的研究工作。

To get more specific on where I think acceleration is likely to come from, a surprisingly large fraction of the progress in biology has come from a truly tiny number of discoveries, often related to broad measurement tools or techniques that allow precise but generalized or programmable intervention in biological systems. There’s perhaps ~1 of these major discoveries per year and collectively they arguably drive >50% of progress in biology. These discoveries are so powerful precisely because they cut through intrinsic complexity and data limitations, directly increasing our understanding and control over biological processes. A few discoveries per decade have enabled both the bulk of our basic scientific understanding of biology, and have driven many of the most powerful medical treatments.
具体来说，我认为加速发展很可能源于一个令人惊讶的事实：生物学领域的进展中，有相当大一部分来自数量极少的重大发现。这些发现通常与广泛的测量工具或技术 相关，它们能对生物系统进行精确但通用或可编程的干预。这类重大发现每年可能仅有约 1 项，但总体而言它们推动了生物学领域超过 50%的进展。这些发现之所以如此强大，正是因为它们突破了固有的复杂性和数据限制，直接增强了我们对生物过程的理解和控制能力。每十年出现的少数几项发现，既构成了我们对生物学基础科学认知的主体，也催生了众多最有效的医疗手段。

Some examples include:一些例子包括：

I’m going to the trouble of listing all these technologies because I want to make a crucial claim about them: I think their rate of discovery could be increased by 10x or more if there were a lot more talented, creative researchers*.* Or, put another way, I think the returns to intelligence are high for these discoveries, and that everything else in biology and medicine mostly follows from them.
我之所以不厌其烦地列举这些技术，是因为我想提出一个关键主张：如果有更多才华横溢、富有创造力的研究人员，我认为这些技术的发现速度可以提升 10 倍甚至更多。换句话说，我认为智力对这些发现的回报率很高，而生物学和医学中的其他进展大多都源于这些发现。

Why do I think this? Because of the answers to some questions that we should get in the habit of asking when we’re trying to determine “returns to intelligence”. First, these discoveries are generally made by a tiny number of researchers, often the same people repeatedly, suggesting skill and not random search (the latter might suggest lengthy experiments are the limiting factor). Second, they often “could have been made” years earlier than they were: for example, CRISPR was a naturally occurring component of the immune system in bacteria that’s been known since the 80’s, but it took another 25 years for people to realize it could be repurposed for general gene editing. They also are often delayed many years by lack of support from the scientific community for promising directions (see this profile on the inventor of mRNA vaccines; similar stories abound). Third, successful projects are often scrappy or were afterthoughts that people didn’t initially think were promising, rather than massively funded efforts. This suggests that it’s not just massive resource concentration that drives discoveries, but ingenuity.
我为何这样认为？因为当我们试图衡量“智力回报”时，应习惯性追问几个问题的答案。首先，这些突破性发现通常由极少数研究者完成，且往往是同一批人反复取得成果，这表明关键因素是技能而非随机探索（后者可能暗示漫长实验是制约因素）。其次，许多发现本可以提前数年问世：例如 CRISPR 作为细菌免疫系统的天然成分自 80 年代就已被发现，但人们又花了 25 年才意识到它能改造为通用基因编辑工具。这些发现还常因科学界对潜力方向的忽视而延迟多年（参见 mRNA 疫苗发明者的遭遇；类似案例比比皆是）。第三，成功项目往往资源匮乏或是起初不被看好的附加研究，而非重金投入的课题。这说明驱动发现的不仅是资源堆砌，更是独创性。

Finally, although some of these discoveries have “serial dependence” (you need to make discovery A first in order to have the tools or knowledge to make discovery B)—which again might create experimental delays—many, perhaps most, are independent, meaning many at once can be worked on in parallel. Both these facts, and my general experience as a biologist, strongly suggest to me that there are hundreds of these discoveries waiting to be made if scientists were smarter and better at making connections between the vast amount of biological knowledge humanity possesses (again consider the CRISPR example). The success of AlphaFold / AlphaProteo at solving important problems much more effectively than humans, despite decades of carefully designed physics modeling, provides a proof of principle (albeit with a narrow tool in a narrow domain) that should point the way forward.
最后，尽管其中一些发现具有“序列依赖性”（需要先有发现 A 才能获得工具或知识来进行发现 B）——这同样可能导致实验延迟——但许多（或许是大多数）发现是独立的，意味着可以同时并行推进多项研究。这些事实，加上我作为生物学家的普遍经验，都强烈表明：如果科学家们更聪明、更善于在人类拥有的海量生物学知识之间建立联系（再次以 CRISPR 为例），还有数百项此类发现正等待被揭示。AlphaFold/AlphaProteo 在解决重要问题上的成功——其效率远超人类数十年精心设计的物理模型——虽局限于狭窄领域的特定工具，却为未来发展路径提供了原则性验证。

Thus, it’s my guess that powerful AI could at least 10x the rate of these discoveries, giving us the next 50-100 years of biological progress in 5-10 years. Why not 100x? Perhaps it is possible, but here both serial dependence and experiment times become important: getting 100 years of progress in 1 year requires a lot of things to go right the first time, including animal experiments and things like designing microscopes or expensive lab facilities. I’m actually open to the (perhaps absurd-sounding) idea that we could get 1000 years of progress in 5-10 years, but very skeptical that we can get 100 years in 1 year. Another way to put it is I think there’s an unavoidable constant delay: experiments and hardware design have a certain “latency” and need to be iterated upon a certain “irreducible” number of times in order to learn things that can’t be deduced logically. But massive parallelism may be possible on top of that.
因此，我猜测强大的人工智能至少能将这类发现的速率提升 10 倍，让我们在 5-10 年内取得相当于未来 50-100 年的生物学进展。 为什么不是 100 倍？或许也有可能，但这里连续依赖性和实验时间都变得很重要：要在 1 年内取得 100 年的进展，很多事情必须第一次就做对，包括动物实验以及设计显微镜或昂贵的实验室设施等。我其实对（可能听起来荒谬的）这个想法持开放态度：我们可能在 5-10 年内取得 1000 年的进展，但对 1 年内取得 100 年进展的说法非常怀疑。另一种说法是，我认为存在一个不可避免的恒定延迟：实验和硬件设计有一定的“延迟”，并且需要“不可减少”的迭代次数，才能学到无法通过逻辑推导获得的知识。但在此基础上，大规模的并行处理或许是可能的 。

What about clinical trials? Although there is a lot of bureaucracy and slowdown associated with them, the truth is that a lot (though by no means all!) of their slowness ultimately derives from the need to rigorously evaluate drugs that barely work or ambiguously work. This is sadly true of most therapies today: the average cancer drug increases survival by a few months while having significant side effects that need to be carefully measured (there’s a similar story for Alzheimer’s drugs). This leads to huge studies (in order to achieve statistical power) and difficult tradeoffs which regulatory agencies generally aren’t great at making, again because of bureaucracy and the complexity of competing interests.
临床试验的情况如何？尽管它们伴随着大量官僚主义和进度迟缓，但事实上，其缓慢很大程度上（尽管绝非全部！）源于需要对那些疗效微弱或模棱两可的药物进行严格评估。遗憾的是，当今大多数疗法皆是如此：平均而言，抗癌药物仅能延长患者数月生存期，却伴随着需要仔细评估的显著副作用（阿尔茨海默病药物也有类似情况）。这导致研究规模庞大（以获得统计效力）以及艰难的权衡取舍，而监管机构通常并不擅长做出这些决策——同样是由于官僚主义和利益博弈的复杂性。

When something works really well, it goes much faster: there’s an accelerated approval track and the ease of approval is much greater when effect sizes are larger. mRNA vaccines for COVID were approved in 9 months—much faster than the usual pace. That said, even under these conditions clinical trials are still too slow—mRNA vaccines arguably should have been approved in ~2 months. But these kinds of delays (~1 year end-to-end for a drug) combined with massive parallelization and the need for some but not too much iteration (“a few tries”) are very compatible with radical transformation in 5-10 years. Even more optimistically, it is possible that AI-enabled biological science will reduce the need for iteration in clinical trials by developing better animal and cell experimental models (or even simulations) that are more accurate in predicting what will happen in humans. This will be particularly important in developing drugs against the aging process, which plays out over decades and where we need a faster iteration loop.
当某样东西效果特别好时，审批进程就会大大加快：效果越显著，审批流程就越容易进入加速通道。新冠 mRNA 疫苗仅用 9 个月就获得批准——远快于常规审批速度。即便如此，在这种特殊情况下临床试验仍然过于缓慢——mRNA 疫苗本可以在约 2 个月内获批。但这类延迟（药物研发端到端耗时约 1 年）结合大规模并行处理能力，以及需要适度而非过多的迭代（"几次尝试"），与 5-10 年内实现彻底变革的愿景高度契合。更乐观地看，AI 赋能的生物科学可能通过开发更精准的动物/细胞实验模型（甚至模拟系统）来减少临床试验迭代需求，这些模型能更准确预测人体反应。这对研发抗衰老药物尤为重要——这个需要数十年观察的领域亟需更快的迭代循环。

Finally, on the topic of clinical trials and societal barriers, it is worth pointing out explicitly that in some ways biomedical innovations have an unusually strong track record of being successfully deployed, in contrast to some other technologies. As mentioned in the introduction, many technologies are hampered by societal factors despite working well technically. This might suggest a pessimistic perspective on what AI can accomplish*.* But biomedicine is unique in that although the process of developing drugs is overly cumbersome, once developed they generally are successfully deployed and used.
最后，关于临床试验和社会障碍的话题，有必要明确指出，在某些方面，生物医学创新有着异常成功的应用记录，与其他一些技术 形成鲜明对比。正如引言中提到的，许多技术在技术上运作良好，却因社会因素而受阻。这可能让人对人工智能的成就持悲观态度。但生物医学的独特之处在于，尽管药物开发过程过于繁琐，但一旦开发成功，通常都能顺利投入使用。

To summarize the above, my basic prediction is that AI-enabled biology and medicine will allow us to compress the progress that human biologists would have achieved over the next 50-100 years into 5-10 years. I’ll refer to this as the “compressed 21st century”: the idea that after powerful AI is developed, we will in a few years make all the progress in biology and medicine that we would have made in the whole 21st century.
综上所述，我的基本预测是：人工智能赋能的生物学和医学将把人类生物学家未来 50-100 年才能实现的进展压缩到 5-10 年内完成。我将其称为“压缩版 21 世纪”——即在强大 AI 诞生后的短短几年内，我们将取得原本需要整个 21 世纪才能完成的生物学与医学进步。

Although predicting what powerful AI can do in a few years remains inherently difficult and speculative, there is some concreteness to asking “what could humans do unaided in the next 100 years?”. Simply looking at what we’ve accomplished in the 20th century, or extrapolating from the first 2 decades of the 21st, or asking what “10 CRISPR’s and 50 CAR-T’s” would get us, all offer practical, grounded ways to estimate the general level of progress we might expect from powerful AI.
尽管预测强大的人工智能在未来几年能做什么本质上仍然困难且充满不确定性，但询问“人类在未来 100 年内不借助外力能实现什么”则显得更为具体。无论是回顾我们在 20 世纪取得的成就，还是从 21 世纪前 20 年进行推断，抑或思考“10 个 CRISPR 和 50 个 CAR-T”能带来什么，这些都为估算强大 AI 可能带来的总体进步水平提供了实际而扎实的方法。

Below I try to make a list of what we might expect. This is not based on any rigorous methodology, and will almost certainly prove wrong in the details, but it’s trying to get across the general level of radicalism we should expect:
以下我尝试列出我们可能预期的事项。这并非基于任何严谨的方法论，细节上几乎肯定会出错，但它试图传达我们应预期的激进程度的大致水平：

It is worth looking at this list and reflecting on how different the world will be if all of it is achieved 7-12 years from now (which would be in line with an aggressive AI timeline). It goes without saying that it would be an unimaginable humanitarian triumph, the elimination all at once of most of the scourges that have haunted humanity for millennia. Many of my friends and colleagues are raising children, and when those children grow up, I hope that any mention of disease will sound to them the way scurvy, smallpox, or bubonic plague sounds to us. That generation will also benefit from increased biological freedom and self-expression, and with luck may also be able to live as long as they want.
审视这份清单并思考，如果这一切在未来 7 到 12 年内实现（这与激进的人工智能发展时间表相符），世界将变得多么不同。毋庸置疑，这将是一场难以想象的人道主义胜利，一举消除困扰人类数千年的多数灾祸。我的许多朋友和同事正在养育子女，当这些孩子长大时，我希望任何关于疾病的提及对他们而言，就像坏血病、天花或黑死病对我们一样陌生。那一代人还将受益于增强的生物自由和自我表达，幸运的话，或许还能随心所欲地延长寿命。

It’s hard to overestimate how surprising these changes will be to everyone except the small community of people who expected powerful AI. For example, thousands of economists and policy experts in the US currently debate how to keep Social Security and Medicare solvent, and more broadly how to keep down the cost of healthcare (which is mostly consumed by those over 70 and especially those with terminal illnesses such as cancer). The situation for these programs is likely to be radically improved if all this comes to pass, as the ratio of working age to retired population will change drastically. No doubt these challenges will be replaced with others, such as how to ensure widespread access to the new technologies, but it is worth reflecting on how much the world will change even if biology is the only area to be successfully accelerated by AI.
除了少数期待强大人工智能出现的人之外，这些变化对所有人来说都将是惊人的，其程度怎么高估都不为过。例如，美国目前有数千名经济学家和政策专家在讨论如何保持社会保障和医疗保险的偿付能力，以及更广泛地说，如何控制医疗保健成本（主要由 70 岁以上人群，尤其是癌症等绝症患者消费）。如果这一切成为现实 ，这些项目的状况可能会得到根本改善，因为工作年龄人口与退休人口的比例将发生巨大变化。毫无疑问，这些挑战将被其他挑战所取代，比如如何确保新技术的广泛普及，但值得思考的是，即使生物学是唯一被人工智能成功加速的领域，世界也将发生多大的变化。

2. Neuroscience and mind 2. 神经科学与心智

In the previous section I focused on physical diseases and biology in general, and didn’t cover neuroscience or mental health. But neuroscience is a subdiscipline of biology and mental health is just as important as physical health. In fact, if anything, mental health affects human well-being even more directly than physical health. Hundreds of millions of people have very low quality of life due to problems like addiction, depression, schizophrenia, low-functioning autism, PTSD, psychopathy, or intellectual disabilities. Billions more struggle with everyday problems that can often be interpreted as much milder versions of one of these severe clinical disorders. And as with general biology, it may be possible to go beyond addressing problems to improving the baseline quality of human experience.
在上一节中，我主要关注了生理疾病和广义的生物学，并未涉及神经科学或心理健康。但神经科学是生物学的分支学科，而心理健康与生理健康同等重要。事实上，心理健康对人类福祉的影响甚至比生理健康更为直接。数以亿计的人因成瘾、抑郁、精神分裂症、低功能自闭症、创伤后应激障碍、精神病态或智力障碍等问题生活质量极低。还有数十亿人日常困扰的问题，往往可被视作这些严重临床疾病的轻度版本。与普通生物学一样，我们或许不仅能解决问题，还能提升人类体验的基准质量。

The basic framework that I laid out for biology applies equally to neuroscience. The field is propelled forward by a small number of discoveries often related to tools for measurement or precise intervention – in the list of those above, optogenetics was a neuroscience discovery, and more recently CLARITY and expansion microscopy are advances in the same vein, in addition to many of the general cell biology methods directly carrying over to neuroscience. I think the rate of these advances will be similarly accelerated by AI and therefore that the framework of “100 years of progress in 5-10 years” applies to neuroscience in the same way it does to biology and for the same reasons. As in biology, the progress in 20th century neuroscience was enormous – for example we didn’t even understand how or why neurons fired until the 1950’s. Thus, it seems reasonable to expect AI-accelerated neuroscience to produce rapid progress over a few years.
我为生物学提出的基本框架同样适用于神经科学。该领域的发展往往由少数关键发现推动，这些发现通常涉及测量工具或精确干预技术——在上述列表中，光遗传学就是神经科学的一项突破，而最近出现的 CLARITY 技术和膨胀显微技术也属于同类进展，此外还有许多通用细胞生物学方法可直接应用于神经科学。我认为人工智能将以类似方式加速这些技术进步，因此"5-10 年内实现百年进展"的框架同样适用于神经科学，其理由与生物学领域完全一致。如同生物学领域，20 世纪神经科学取得了巨大进步——例如直到 1950 年代我们才真正理解神经元如何及为何放电。因此，我们有理由预期人工智能加持的神经科学将在未来数年内实现快速突破。

There is one thing we should add to this basic picture, which is that some of the things we’ve learned (or are learning) about AI itself in the last few years are likely to help advance neuroscience, even if it continues to be done only by humans. Interpretability is an obvious example: although biological neurons superficially operate in a completely different manner from artificial neurons (they communicate via spikes and often spike rates, so there is a time element not present in artificial neurons, and a bunch of details relating to cell physiology and neurotransmitters modifies their operation substantially), the basic question of “how do distributed, trained networks of simple units that perform combined linear/non-linear operations work together to perform important computations” is the same, and I strongly suspect the details of individual neuron communication will be abstracted away in most of the interesting questions about computation and circuits. As just one example of this, a computational mechanism discovered by interpretability researchers in AI systems was recently rediscovered in the brains of mice.
我们还应在这个基本框架上补充一点：过去几年中，我们对人工智能本身的一些认识（或正在认识的内容）很可能推动神经科学的发展，即便这些研究仍仅由人类完成。可解释性就是个明显的例子：虽然生物神经元表面上以与人工神经元完全不同的方式运作（它们通过脉冲和脉冲频率进行通信，因此存在人工神经元所不具备的时间要素，且细胞生理学和神经递质等大量细节会显著改变其运作方式），但"执行线性/非线性组合操作的分布式训练简单单元网络如何协同完成重要计算"这一基本问题是相同的。我强烈怀疑，在大多数关于计算与神经回路的有趣问题中，单个神经元通信的细节将被抽象化 。仅举一例：人工智能可解释性研究者在 AI 系统中发现的计算机制，最近在小鼠大脑中被重新发现。

It is much easier to do experiments on artificial neural networks than on real ones (the latter often requires cutting into animal brains), so interpretability may well become a tool for improving our understanding of neuroscience. Furthermore, powerful AI’s will themselves probably be able to develop and apply this tool better than humans can.
在人工神经网络上进行实验比在真实神经网络上容易得多（后者通常需要切开动物大脑），因此可解释性很可能成为增进我们对神经科学理解的工具。此外，强大的人工智能本身可能比人类更擅长开发和应用这一工具。

Beyond just interpretability though, what we have learned from AI about how intelligent systems are trained should (though I am not sure it has yet) cause a revolution in neuroscience. When I was working in neuroscience, a lot of people focused on what I would now consider the wrong questions about learning, because the concept of the scaling hypothesis / bitter lesson didn’t exist yet. The idea that a simple objective function plus a lot of data can drive incredibly complex behaviors makes it more interesting to understand the objective functions and architectural biases and less interesting to understand the details of the emergent computations. I have not followed the field closely in recent years, but I have a vague sense that computational neuroscientists have still not fully absorbed the lesson. My attitude to the scaling hypothesis has always been “aha – this is an explanation, at a high level, of how intelligence works and how it so easily evolved”, but I don’t think that’s the average neuroscientist’s view, in part because the scaling hypothesis as “the secret to intelligence” isn’t fully accepted even within AI.
不过，除了可解释性之外，我们从人工智能中学到的关于智能系统如何训练的知识，本应（尽管我不确定是否已经）引发神经科学领域的一场革命。当我从事神经科学研究时，许多人关注的是我现在认为关于学习的错误问题，因为当时还不存在扩展假说/苦涩教训的概念。一个简单的目标函数加上大量数据可以驱动极其复杂的行为，这一观点使得理解目标函数和架构偏好变得更有趣，而对涌现计算的细节理解则显得不那么重要。近年来我没有紧密跟踪这一领域，但我隐约感觉到计算神经科学家们仍未完全吸收这一教训。我对扩展假说的态度一直是“啊哈——这是对智能如何运作以及它为何如此容易进化的一种高层次解释”，但我不认为这是普通神经科学家的观点，部分原因在于，即使是在人工智能领域内，“扩展假说作为智能的秘诀”也尚未被完全接受。

I think that neuroscientists should be trying to combine this basic insight with the particularities of the human brain (biophysical limitations, evolutionary history, topology, details of motor and sensory inputs/outputs) to try to figure out some of neuroscience’s key puzzles. Some likely are, but I suspect it’s not enough yet, and that AI neuroscientists will be able to more effectively leverage this angle to accelerate progress.
我认为神经科学家应该尝试将这一基本见解与人类大脑的特殊性（生物物理限制、进化历史、拓扑结构、运动和感觉输入/输出的细节）结合起来，以试图解决神经科学中的一些关键难题。有些人可能已经在这样做，但我怀疑目前还不够，而人工智能神经科学家将能够更有效地利用这一角度来加速进展。

I expect AI to accelerate neuroscientific progress along four distinct routes, all of which can hopefully work together to cure mental illness and improve function:
我预计人工智能将通过四条不同的途径加速神经科学的进步，所有这些途径有望共同作用，治愈精神疾病并提升功能：

It’s my guess that these four routes of progress working together would, as with physical disease, be on track to lead to the cure or prevention of most mental illness in the next 100 years even if AI was not involved – and thus might reasonably be completed in 5-10 AI-accelerated years. Concretely my guess at what will happen is something like:
我猜测，即使没有人工智能的参与，这四种进步途径协同作用，就像治疗身体疾病一样，有望在未来 100 年内治愈或预防大多数精神疾病——因此在人工智能加速下，可能只需 5 到 10 年就能合理完成。具体而言，我对未来发展的预测大致如下：

One topic that often comes up in sci-fi depictions of AI, but that I intentionally haven’t discussed here, is “mind uploading”, the idea of capturing the pattern and dynamics of a human brain and instantiating them in software. This topic could be the subject of an essay all by itself, but suffice it to say that while I think uploading is almost certainly possible in principle, in practice it faces significant technological and societal challenges, even with powerful AI, that likely put it outside the 5-10 year window we are discussing.
在科幻作品中经常出现的一个话题是“意识上传”，即捕捉人类大脑的模式和动态并在软件中实例化的想法。虽然我刻意没有在这里讨论这一点，但这个话题本身就可以单独写一篇文章。简而言之，我认为上传在原则上几乎肯定是可行的，但在实践中，即使有强大的人工智能，它也面临着巨大的技术和社会挑战，很可能超出了我们正在讨论的 5-10 年时间范围。

In summary, AI-accelerated neuroscience is likely to vastly improve treatments for, or even cure, most mental illness as well as greatly expand “cognitive and mental freedom” and human cognitive and emotional abilities. It will be every bit as radical as the improvements in physical health described in the previous section. Perhaps the world will not be visibly different on the outside, but the world as experienced by humans will be a much better and more humane place, as well as a place that offers greater opportunities for self-actualization. I also suspect that improved mental health will ameliorate a lot of other societal problems, including ones that seem political or economic.
总之，AI 加速的神经科学研究有望极大改善甚至治愈大多数精神疾病，同时大幅扩展“认知与心智自由”及人类的情感和认知能力。其变革性将丝毫不亚于前文所述生理健康领域的进步。或许从外表看世界并无明显变化，但人类所体验的世界将变得更加美好、更富人性，也为自我实现提供了更多可能。我推测，改善的心理健康还将缓解许多其他社会问题，包括那些看似政治或经济层面的问题。

3. Economic development and poverty3. 经济发展与贫困问题

The previous two sections are about developing new technologies that cure disease and improve the quality of human life. However an obvious question, from a humanitarian perspective, is: “will everyone have access to these technologies?”
前两节讨论了治愈疾病并提升人类生活质量的新技术开发。但从人道主义视角出发，一个显而易见的问题是：“所有人都能获得这些技术吗？”

It is one thing to develop a cure for a disease, it is another thing to eradicate the disease from the world. More broadly, many existing health interventions have not yet been applied everywhere in the world, and for that matter the same is true of (non-health) technological improvements in general. Another way to say this is that living standards in many parts of the world are still desperately poor: GDP per capita is ~$2,000 in Sub-Saharan Africa as compared to ~$75,000 in the United States. If AI further increases economic growth and quality of life in the developed world, while doing little to help the developing world, we should view that as a terrible moral failure and a blemish on the genuine humanitarian victories in the previous two sections. Ideally, powerful AI should help the developing world catch up to the developed world, even as it revolutionizes the latter.
研发一种疾病的治疗方法是一回事，将其从世界上根除则是另一回事。更广泛地说，许多现有的健康干预措施尚未在世界各地得到应用，同样，（非健康领域的）技术改进总体上也是如此。换句话说，世界上许多地区的生活水平仍然极度贫困：撒哈拉以南非洲的人均 GDP 约为 2000 美元，而美国约为 75000 美元。如果人工智能进一步提高了发达国家的经济增长和生活质量，而对发展中国家帮助甚微，我们应将其视为一种可怕的道德失败，也是前两部分所述真正人道主义胜利的污点。理想情况下，强大的人工智能应该在彻底改变发达国家的同时，帮助发展中国家迎头赶上。

I am not as confident that AI can address inequality and economic growth as I am that it can invent fundamental technologies, because technology has such obvious high returns to intelligence (including the ability to route around complexities and lack of data) whereas the economy involves a lot of constraints from humans, as well as a large dose of intrinsic complexity. I am somewhat skeptical that an AI could solve the famous “ socialist calculation problem ” and I don’t think governments will (or should) turn over their economic policy to such an entity, even if it could do so. There are also problems like how to convince people to take treatments that are effective but that they may be suspicious of.
对于人工智能能否解决不平等和经济增长问题，我不像对它能发明基础技术那样有信心，因为技术在智力回报上（包括绕过复杂性和数据不足的能力）具有显而易见的高收益，而经济则涉及大量来自人类的约束，以及内在的复杂性。我对人工智能能否解决著名的“社会主义计算问题”持一定怀疑态度 ，并且我不认为政府会（或应该）将经济政策交给这样的实体，即使它能够做到。此外，还存在如何说服人们接受有效但可能心存疑虑的治疗方案等问题。

The challenges facing the developing world are made even more complicated by pervasive corruption in both private and public sectors. Corruption creates a vicious cycle: it exacerbates poverty, and poverty in turn breeds more corruption. AI-driven plans for economic development need to reckon with corruption, weak institutions, and other very human challenges.
发展中国家面临的挑战因私营和公共部门普遍存在的腐败而变得更加复杂。腐败形成了一个恶性循环：它加剧了贫困，而贫困又滋生了更多的腐败。人工智能驱动的经济发展计划需要考虑到腐败、制度薄弱以及其他非常人性化的挑战。

Nevertheless, I do see significant reasons for optimism. Diseases have been eradicated and many countries have gone from poor to rich, and it is clear that the decisions involved in these tasks exhibit high returns to intelligence (despite human constraints and complexity). Therefore, AI can likely do them better than they are currently being done. There may also be targeted interventions that get around the human constraints and that AI could focus on. More importantly though, we have to try. Both AI companies and developed world policymakers will need to do their part to ensure that the developing world is not left out; the moral imperative is too great. So in this section, I’ll continue to make the optimistic case, but keep in mind everywhere that success is not guaranteed and depends on our collective efforts.
尽管如此，我确实看到了乐观的重要理由。疾病已被根除，许多国家从贫困走向富裕，显然在这些任务中做出的决策体现了智力带来的高回报（尽管存在人类的局限性和复杂性）。因此，人工智能很可能比目前人类做得更好。或许还存在一些针对性干预措施，能够绕过人类的限制，这正是人工智能可以专注的方向。但更重要的是，我们必须尝试。无论是 AI 公司还是发达国家的政策制定者，都需要尽自己的一份力，确保发展中世界不被落下；道德上的迫切性不容忽视。因此，在这一部分，我将继续阐述乐观的理由，但请时刻谨记，成功并非必然，它取决于我们共同的努力。

Below I make some guesses about how I think things may go in the developing world over the 5-10 years after powerful AI is developed:
以下是我对未来 5 到 10 年内，强大的人工智能出现后，发展中世界可能发生的变化的一些猜测：

Overall, I am optimistic about quickly bringing AI’s biological advances to people in the developing world. I am hopeful, though not confident, that AI can also enable unprecedented economic growth rates and allow the developing world to at least surpass where the developed world is now. I am concerned about the “opt out” problem in both the developed and developing world, but suspect that it will peter out over time and that AI can help accelerate this process. It won’t be a perfect world, and those who are behind won’t fully catch up, at least not in the first few years. But with strong efforts on our part, we may be able to get things moving in the right direction—and fast. If we do, we can make at least a downpayment on the promises of dignity and equality that we owe to every human being on earth.
总体而言，我对迅速将人工智能在生物领域的进步带给发展中国家人民持乐观态度。我虽不确定但抱有希望，认为 AI 还能实现前所未有的经济增长率，使发展中国家至少超越发达国家目前的水平。我担忧发达国家和发展中国家都会面临的"退出选择"问题，但推测随着时间推移该问题会逐渐消退，而 AI 有助于加速这一进程。未来不会是一个完美的世界，落后群体无法完全赶上，至少在最初几年内如此。但通过我们的大力推动，或许能让事态朝正确方向快速前进。若能实现，我们至少能为兑现地球上每个人应享有的尊严与平等承诺支付首期款。

4. Peace and governance 4. 和平与治理

Suppose that everything in the first three sections goes well: disease, poverty, and inequality are significantly reduced and the baseline of human experience is raised substantially. It does not follow that all major causes of human suffering are solved. Humans are still a threat to each other*.* Although there is a trend of technological improvement and economic development leading to democracy and peace, it is a very loose trend, with frequent (and recent) backsliding. At the dawn of the 20th Century, people thought they had put war behind them; then came the two world wars. Thirty years ago Francis Fukuyama wrote about “ the End of History ” and a final triumph of liberal democracy; that hasn’t happened yet. Twenty years ago US policymakers believed that free trade with China would cause it to liberalize as it became richer; that very much didn’t happen, and we now seem headed for a second cold war with a resurgent authoritarian bloc. And plausible theories suggest that internet technology may actually advantage authoritarianism, not democracy as initially believed (e.g. in the “Arab Spring” period). It seems important to try to understand how powerful AI will intersect with these issues of peace, democracy, and freedom.
假设前三部分所述的一切进展顺利：疾病、贫困和不平等现象显著减少，人类生活的基本水平得到大幅提升。但这并不意味着所有导致人类苦难的主要根源都已解决。人类彼此间仍构成威胁。尽管技术进步与经济发展推动民主与和平的趋势存在，但这种趋势极为脆弱，常有（甚至近期）倒退现象。20 世纪初，人们曾以为战争已成为过去；随后却爆发了两次世界大战。三十年前，弗朗西斯·福山提出"历史的终结"和自由民主的最终胜利；这一预言至今未实现。二十年前，美国政策制定者认为与中国的自由贸易会使其在富裕过程中走向自由化；事实恰恰相反，现在我们似乎正走向与复兴的威权集团之间的第二次冷战。而有说服力的理论指出，互联网技术实际上可能助长威权主义，而非如最初（如"阿拉伯之春"时期）所认为的那样有利于民主。 试图理解强大的人工智能将如何与和平、民主和自由这些议题相交织，似乎至关重要。

Unfortunately, I see no strong reason to believe AI will preferentially or structurally advance democracy and peace, in the same way that I think it will structurally advance human health and alleviate poverty. Human conflict is adversarial and AI can in principle help both the “good guys” and the “bad guys”. If anything, some structural factors seem worrying: AI seems likely to enable much better propaganda and surveillance, both major tools in the autocrat’s toolkit. It’s therefore up to us as individual actors to tilt things in the right direction: if we want AI to favor democracy and individual rights, we are going to have to fight for that outcome. I feel even more strongly about this than I do about international inequality: the triumph of liberal democracy and political stability is not guaranteed, perhaps not even likely, and will require great sacrifice and commitment on all of our parts, as it often has in the past.
不幸的是，我看不到强有力的理由相信人工智能会优先或在结构上促进民主与和平，就像我认为它会在结构上改善人类健康和减轻贫困那样。人类冲突具有对抗性，原则上 AI 既能帮助“好人”，也能帮助“坏人”。甚至某些结构性因素令人担忧：AI 似乎可能大幅提升宣传与监控能力，而这两者正是独裁者工具箱中的主要手段。因此，要靠我们作为个体行动者将局势推向正确方向：如果我们希望 AI 有利于民主和个人权利，就必须为之奋斗。对此我的信念甚至比对国际不平等问题更强烈：自由民主与政治稳定的胜利并非必然，或许甚至可能性不大，需要我们所有人付出巨大牺牲和坚定承诺，正如历史上屡见不鲜的那样。

I think of the issue as having two parts: international conflict, and the internal structure of nations. On the international side, it seems very important that democracies have the upper hand on the world stage when powerful AI is created. AI-powered authoritarianism seems too terrible to contemplate, so democracies need to be able to set the terms by which powerful AI is brought into the world, both to avoid being overpowered by authoritarians and to prevent human rights abuses within authoritarian countries.
我认为这个问题可以分为两部分：国际冲突与国家内部结构。在国际层面，当强大的人工智能被创造出来时，民主国家在世界舞台上占据优势显得至关重要。由人工智能驱动的威权主义可怕到令人不敢想象，因此民主国家必须有能力制定强大 AI 问世的基本规则——既要避免被威权势力压制，也要防止威权国家内部发生人权侵害。

My current guess at the best way to do this is via an “entente strategy”, in which a coalition of democracies seeks to gain a clear advantage (even just a temporary one) on powerful AI by securing its supply chain, scaling quickly, and blocking or delaying adversaries’ access to key resources like chips and semiconductor equipment. This coalition would on one hand use AI to achieve robust military superiority (the stick) while at the same time offering to distribute the benefits of powerful AI (the carrot) to a wider and wider group of countries in exchange for supporting the coalition’s strategy to promote democracy (this would be a bit analogous to “ Atoms for Peace ”). The coalition would aim to gain the support of more and more of the world, isolating our worst adversaries and eventually putting them in a position where they are better off taking the same bargain as the rest of the world: give up competing with democracies in order to receive all the benefits and not fight a superior foe.
我目前猜测的最佳方式是通过“协约战略” ，即民主国家联盟通过确保供应链、快速扩展以及阻止或延缓对手获取芯片和半导体设备等关键资源，在强大 AI 领域取得明显优势（哪怕是暂时的）。该联盟一方面将利用 AI 实现稳固的军事优势（大棒），同时向越来越多的国家提供强大 AI 的收益（胡萝卜），以换取它们支持联盟促进民主的战略（这有点类似于“原子能为和平”计划）。联盟的目标是争取越来越多国家的支持，孤立我们最恶劣的对手，最终使他们处于更有利的位置，接受与世界其他国家相同的交易：放弃与民主国家竞争，以换取所有利益，不与更强大的对手对抗。

If we can do all this, we will have a world in which democracies lead on the world stage and have the economic and military strength to avoid being undermined, conquered, or sabotaged by autocracies, and may be able to parlay their AI superiority into a durable advantage. This could optimistically lead to an “eternal 1991”—a world where democracies have the upper hand and Fukuyama’s dreams are realized. Again, this will be very difficult to achieve, and will in particular require close cooperation between private AI companies and democratic governments, as well as extraordinarily wise decisions about the balance between carrot and stick.
如果我们能做到这一切，我们将迎来一个民主国家在世界舞台上占据主导地位的世界，它们拥有足够的经济和军事实力，避免被专制国家削弱、征服或破坏，并可能将其在人工智能领域的优势转化为持久的优势。乐观地说，这可能会带来一个“永恒的 1991 年”——一个民主国家占据上风、福山梦想得以实现的世界。再次强调，这将非常难以实现，尤其需要私营人工智能公司与民主政府之间的密切合作，以及在胡萝卜加大棒之间做出极其明智的平衡决策。

Even if all that goes well, it leaves the question of the fight between democracy and autocracy within each country. It is obviously hard to predict what will happen here, but I do have some optimism that given a global environment in which democracies control the most powerful AI, then AI may actually structurally favor democracy everywhere. In particular, in this environment democratic governments can use their superior AI to win the information war: they can counter influence and propaganda operations by autocracies and may even be able to create a globally free information environment by providing channels of information and AI services in a way that autocracies lack the technical ability to block or monitor. It probably isn’t necessary to deliver propaganda, only to counter malicious attacks and unblock the free flow of information. Although not immediate, a level playing field like this stands a good chance of gradually tilting global governance towards democracy, for several reasons.
即便一切进展顺利，各国国内民主与专制的斗争问题依然存在。虽然难以预测具体走向，但我对以下情况持乐观态度：若全球环境中民主国家掌控最强大的人工智能，那么 AI 可能会在结构上普遍有利于民主体制。尤其在这种环境下，民主政府能凭借其先进的 AI 技术赢得信息战——它们可以反击专制政权的影响力操纵和宣传行动，甚至可能通过建立信息渠道和提供 AI 服务（专制国家因技术限制无法封锁或监控这些服务），创造全球自由的信息环境。或许无需主动输出宣传内容，只需抵御恶意攻击并保障信息自由流通。尽管效果不会立竿见影，但这样的公平竞争环境有望基于多重原因，逐步推动全球治理向民主方向倾斜。

First, the increases in quality of life in Sections 1-3 should, all things equal, promote democracy: historically they have, to at least some extent. In particular I expect improvements in mental health, well-being, and education to increase democracy, as all three are negatively with support for authoritarian leaders. In general people want more self-expression when their other needs are met, and democracy is among other things a form of self-expression. Conversely, authoritarianism thrives on fear and resentment.
首先，在其他条件相同的情况下，第 1-3 节中生活质量的提高应会促进民主：从历史上看，它们至少在某种程度上起到了这种作用。特别是，我预计心理健康、幸福感和教育方面的改善将增加民主，因为这三者都与对威权领导人的支持呈负相关。一般来说，当人们的其他需求得到满足时，他们会渴望更多的自我表达，而民主在某种程度上就是一种自我表达的形式。相反，威权主义则依赖于恐惧和怨恨而兴盛。

Second, there is a good chance free information really does undermine authoritarianism, as long as the authoritarians can’t censor it. And uncensored AI can also bring individuals powerful tools for undermining repressive governments. Repressive governments survive by denying people a certain kind of common knowledge, keeping them from realizing that “the emperor has no clothes”. For example Srđa Popović, who helped to topple the Milošević government in Serbia, has written extensively about techniques for psychologically robbing authoritarians of their power, for breaking the spell and rallying support against a dictator. A superhumanly effective AI version of Popović (whose skills seem like they have high returns to intelligence) in everyone’s pocket, one that dictators are powerless to block or censor, could create a wind at the backs of dissidents and reformers across the world. To say it again, this will be a long and protracted fight, one where victory is not assured, but if we design and build AI in the right way, it may at least be a fight where the advocates of freedom everywhere have an advantage.
其次，只要威权政府无法审查信息，自由信息确实有很大可能削弱威权主义。未经审查的人工智能还能为个人提供强大工具来瓦解压迫性政府。压迫性政府赖以生存的手段是剥夺民众的某种共识，阻止他们意识到"皇帝的新装"。例如，曾协助推翻塞尔维亚米洛舍维奇政权的斯尔贾·波波维奇，就大量著述过如何从心理上剥夺威权者权力、打破魔咒并集结力量反抗独裁者的技巧。若每个人口袋中都拥有一个超级智能版的波波维奇（其技巧似乎对智力要求极高），而独裁者又无力封锁或审查，这将成为全球异见者和改革者背后的助力。再次强调，这将是一场漫长而持久的斗争，胜负未卜，但如果我们以正确方式设计和构建人工智能，至少能让世界各地的自由倡导者占据优势。

As with neuroscience and biology, we can also ask how things could be “better than normal”—not just how to avoid autocracy, but how to make democracies better than they are today. Even within democracies, injustices happen all the time. Rule-of-law societies make a promise to their citizens that everyone will be equal under the law and everyone is entitled to basic human rights, but obviously people do not always receive those rights in practice. That this promise is even partially fulfilled makes it something to be proud of, but can AI help us do better?
如同在神经科学和生物学领域一样，我们也可以探讨如何让事物“比正常更好”——不仅仅是避免专制，而是如何让民主比今天更加完善。即使在民主国家中，不公正现象也时有发生。法治社会向公民承诺法律面前人人平等，人人享有基本人权，但显然在实践中人们并不总能获得这些权利。这一承诺即使部分实现也值得自豪，但人工智能能否帮助我们做得更好？

For example, could AI improve our legal and judicial system by making decisions and processes more impartial? Today people mostly worry in legal or judicial contexts that AI systems will be a cause of discrimination, and these worries are important and need to be defended against. At the same time, the vitality of democracy depends on harnessing new technologies to improve democratic institutions, not just responding to risks. A truly mature and successful implementation of AI has the potential to reduce bias and be fairer for everyone.
例如，人工智能能否通过使决策和程序更加公正来改进我们的法律和司法体系？如今在法律或司法背景下，人们主要担忧人工智能系统会成为歧视的根源，这些担忧至关重要，需要加以防范。与此同时，民主的活力在于利用新技术改善民主制度，而不仅仅是应对风险。一个真正成熟且成功实施的人工智能系统，有潜力减少偏见，为所有人带来更公平的结果。

For centuries, legal systems have faced the dilemma that the law aims to be impartial, but is inherently subjective and thus must be interpreted by biased humans. Trying to make the law fully mechanical hasn’t worked because the real world is messy and can’t always be captured in mathematical formulas. Instead legal systems rely on notoriously imprecise criteria like “ cruel and unusual punishment ” or “ utterly without redeeming social importance ”, which humans then interpret—and often do so in a manner that displays bias, favoritism, or arbitrariness. “ Smart contracts ” in cryptocurrencies haven’t revolutionized law because ordinary code isn’t smart enough to adjudicate all that much of interest. But AI might be smart enough for this: it is the first technology capable of making broad, fuzzy judgements in a repeatable and mechanical way.
几个世纪以来，法律体系一直面临着一个困境：法律旨在公正，但本质上具有主观性，因此必须由带有偏见的人类来诠释。试图使法律完全机械化并未奏效，因为现实世界错综复杂，无法总是用数学公式来概括。相反，法律体系依赖于众所周知的模糊标准，如“残酷且不寻常的惩罚”或“完全缺乏救赎社会价值”，然后由人类进行解读——而这种解读往往显示出偏见、偏袒或武断。加密货币中的“智能合约”并未彻底改变法律，因为普通代码的智能程度不足以裁决太多有价值的内容。但人工智能可能足够聪明来完成这一任务：它是第一种能够以可重复且机械化的方式做出广泛、模糊判断的技术。

I am not suggesting that we literally replace judges with AI systems, but the combination of impartiality with the ability to understand and process messy, real world situations feels like it should have some serious positive applications to law and justice. At the very least, such systems could work alongside humans as an aid to decision-making. Transparency would be important in any such system, and a mature science of AI could conceivably provide it: the training process for such systems could be extensively studied, and advanced interpretability techniques could be used to see inside the final model and assess it for hidden biases, in a way that is simply not possible with humans. Such AI tools could also be used to monitor for violations of fundamental rights in a judicial or police context, making constitutions more self-enforcing.
我并非主张直接用 AI 系统取代法官，但这种结合了公正性且能理解处理复杂现实情境的能力，理应能在法律与司法领域产生重大积极应用。至少，这类系统可以作为人类决策的辅助工具。任何此类系统的透明度都至关重要，而成熟的 AI 科学有望实现这一点：通过深入研究系统训练过程，运用先进的可解释性技术透视最终模型并评估潜在偏见——这种透明度在人类决策中根本无法实现。此类 AI 工具还能用于监控司法或警务场景中的基本权利侵害行为，使宪法更具自我执行效力。

In a similar vein, AI could be used to both aggregate opinions and drive consensus among citizens, resolving conflict, finding common ground, and seeking compromise. Some early ideas in this direction have been undertaken by the computational democracy project, including collaborations with Anthropic. A more informed and thoughtful citizenry would obviously strengthen democratic institutions.
同样地，人工智能可用于汇总意见并推动公民达成共识，解决冲突、寻找共同点并寻求妥协。计算民主项目已在这一方向上开展了一些初步构想，包括与 Anthropic 的合作。一个更知情、更深思熟虑的公民群体显然会加强民主制度。

There is also a clear opportunity for AI to be used to help provision government services—such as health benefits or social services—that are in principle available to everyone but in practice often severely lacking, and worse in some places than others. This includes health services, the DMV, taxes, social security, building code enforcement, and so on. Having a very thoughtful and informed AI whose job is to give you everything you’re legally entitled to by the government in a way you can understand—and who also helps you comply with often confusing government rules—would be a big deal. Increasing state capacity both helps to deliver on the promise of equality under the law, and strengthens respect for democratic governance. Poorly implemented services are currently a major driver of cynicism about government.
人工智能在协助提供政府服务方面也展现出明确机遇——诸如健康福利或社会服务等原则上人人可享、但实际上常严重不足且地区差异显著的服务领域。这涵盖医疗服务、车管所事务、税务、社会保障、建筑规范执行等诸多方面。设想一个深思熟虑且信息完备的 AI 系统，其职责是以易懂方式确保民众获得依法享有的全部政府权益，同时协助应对晦涩的行政法规——这将具有重大意义。提升政府执行力既能兑现法律面前人人平等的承诺，又可增强对民主治理的尊重。当前服务实施不力正是催生民众对政府冷嘲热讽的主要诱因 。

All of these are somewhat vague ideas, and as I said at the beginning of this section, I am not nearly as confident in their feasibility as I am in the advances in biology, neuroscience, and poverty alleviation. They may be unrealistically utopian. But the important thing is to have an ambitious vision, to be willing to dream big and try things out. The vision of AI as a guarantor of liberty, individual rights, and equality under the law is too powerful a vision not to fight for. A 21st century, AI-enabled polity could be both a stronger protector of individual freedom, and a beacon of hope that helps make liberal democracy the form of government that the whole world wants to adopt.
所有这些想法都有些模糊，正如我在本节开头所说，我对它们的可行性远不如对生物学、神经科学和扶贫进展那样有信心。它们可能是不切实际的乌托邦。但重要的是要有一个雄心勃勃的愿景，敢于大胆梦想并勇于尝试。将 AI 视为自由、个人权利和法律平等保障者的愿景如此强大，值得我们为之奋斗。一个由 AI 赋能的 21 世纪政体，既能成为个人自由的更强有力守护者，也能成为希望的灯塔，帮助自由民主制成为全世界都渴望采用的政府形式。

5. Work and meaning 5. 工作与意义

Even if everything in the preceding four sections goes well—not only do we alleviate disease, poverty, and inequality, but liberal democracy becomes the dominant form of government, and existing liberal democracies become better versions of themselves—at least one important question still remains. “It’s great we live in such a technologically advanced world as well as a fair and decent one”, someone might object, “but with AI’s doing everything, how will humans have meaning? For that matter, how will they survive economically?”.
即使前四部分所述的一切进展顺利——我们不仅消除了疾病、贫困和不平等，使自由民主成为主导的政府形式，现有的自由民主国家也实现了自我完善——至少还有一个重要问题悬而未决。"我们生活在这样一个技术高度发达且公平体面的世界固然美好，"有人或许会质疑，"但如果 AI 包揽一切，人类将如何获得意义？更现实地说，他们如何维持经济生存？"

I think this question is more difficult than the others. I don’t mean that I am necessarily more pessimistic about it than I am about the other questions (although I do see challenges). I mean that it is fuzzier and harder to predict in advance, because it relates to macroscopic questions about how society is organized that tend to resolve themselves only over time and in a decentralized manner. For example, historical hunter-gatherer societies might have imagined that life is meaningless without hunting and various kinds of hunting-related religious rituals, and would have imagined that our well-fed technological society is devoid of purpose. They might also have not understood how our economy can provide for everyone, or what function people can usefully service in a mechanized society.
我认为这个问题比其他问题更为棘手。我并不是说我对这个问题的看法比其他问题更悲观（尽管我确实看到了挑战），而是说它更加模糊，更难提前预测，因为它涉及到关于社会如何组织等宏观问题，这些问题往往需要时间和去中心化的方式自行解决。例如，历史上的狩猎采集社会可能认为，没有狩猎和各种与狩猎相关的宗教仪式，生活就毫无意义，并可能认为我们丰衣足食的技术社会缺乏目的。他们也可能无法理解我们的经济如何能满足每个人的需求，或者在机械化社会中人们能发挥什么有用的作用。

Nevertheless, it’s worth saying at least a few words, while keeping in mind that the brevity of this section is not at all to be taken as a sign that I don’t take these issues seriously—on the contrary, it is a sign of a lack of clear answers.
尽管如此，还是值得说上几句，同时要记住，本节篇幅简短绝不意味着我不重视这些问题——恰恰相反，这正说明目前缺乏明确的答案。

On the question of meaning, I think it is very likely a mistake to believe that tasks you undertake are meaningless simply because an AI could do them better. Most people are not the best in the world at anything, and it doesn’t seem to bother them particularly much. Of course today they can still contribute through comparative advantage, and may derive meaning from the economic value they produce, but people also greatly enjoy activities that produce no economic value. I spend plenty of time playing video games, swimming, walking around outside, and talking to friends, all of which generates zero economic value. I might spend a day trying to get better at a video game, or faster at biking up a mountain, and it doesn’t really matter to me that someone somewhere is much better at those things. In any case I think meaning comes mostly from human relationships and connection, not from economic labor. People do want a sense of accomplishment, even a sense of competition, and in a post-AI world it will be perfectly possible to spend years attempting some very difficult task with a complex strategy, similar to what people do today when they embark on research projects, try to become Hollywood actors, or found companies. The facts that (a) an AI somewhere could in principle do this task better, and (b) this task is no longer an economically rewarded element of a global economy, don’t seem to me to matter very much.
关于意义的问题，我认为人们很容易陷入一种误区——仅仅因为人工智能能更出色地完成某项任务，就认定自己从事的工作毫无意义。事实上，绝大多数人并非任何领域的顶尖高手，但这似乎并未给他们带来太多困扰。当然，如今人们仍能通过比较优势作出贡献，并可能从创造的经济价值中获得意义，但人类同样热衷于那些不产生经济价值的活动。我花大量时间打电子游戏、游泳、户外散步、与朋友聊天，这些活动完全不创造经济价值。或许我会花一整天提升游戏技巧，或是练习更快地骑车上山，即便世上有人在这些事上远胜于我，对我而言也无关紧要。归根结底，我认为意义主要源自人际关系与情感联结，而非经济性劳动。 人们确实渴望成就感，甚至竞争感，在后 AI 时代，完全有可能花费数年时间尝试完成某项需要复杂策略的艰巨任务，就像如今人们投身研究项目、试图成为好莱坞演员或创办公司那样。关键在于：(a) 某处的人工智能原则上能更出色地完成这项任务，(b) 该任务不再是全球经济中受经济回报的组成部分——在我看来，这些事实并不那么重要。

The economic piece actually seems more difficult to me than the meaning piece. By “economic” in this section I mean the possible problem that most or all humans may not be able to contribute meaningfully to a sufficiently advanced AI-driven economy. This is a more macro problem than the separate problem of inequality, especially inequality in access to the new technologies, which I discussed in Section 3.
经济层面的问题在我看来实际上比意义层面的更棘手。本节所说的"经济"问题，指的是在高度发达的人工智能驱动型经济中，大多数人可能无法做出有意义贡献的潜在困境。这比第三节讨论的不平等问题（尤其是新技术获取机会的不平等）更具宏观性。

First of all, in the short term I agree with arguments that comparative advantage will continue to keep humans relevant and in fact increase their productivity, and may even in some ways level the playing field between humans. As long as AI is only better at 90% of a given job, the other 10% will cause humans to become highly leveraged, increasing compensation and in fact creating a bunch of new human jobs complementing and amplifying what AI is good at, such that the “10%” expands to continue to employ almost everyone. In fact, even if AI can do 100% of things better than humans, but it remains inefficient or expensive at some tasks, or if the resource inputs to humans and AI’s are meaningfully different, then the logic of comparative advantage continues to apply. One area humans are likely to maintain a relative (or even absolute) advantage for a significant time is the physical world. Thus, I think that the human economy may continue to make sense even a little past the point where we reach “a country of geniuses in a datacenter”.
首先，短期内我认同比较优势将继续保持人类的重要性，甚至提升其生产力，并在某些方面可能拉平人类之间的竞争环境。只要人工智能仅在 90%的工作上表现更优，剩下的 10%将使人类变得极具杠杆效应，提高报酬，并实际上创造一系列新的人类岗位，补充和放大 AI 的擅长领域，使得这“10%”不断扩展，持续雇佣几乎所有人。事实上，即便 AI 能在 100%的事情上超越人类，但某些任务上效率低下或成本高昂，或者人类与 AI 的资源投入存在显著差异，比较优势的逻辑依然适用。人类可能长期保持相对（甚至绝对）优势的一个领域是物理世界。因此，我认为人类经济在“数据中心里的天才国度”阶段之后仍可能持续存在意义。

However, I do think in the long run AI will become so broadly effective and so cheap that this will no longer apply. At that point our current economic setup will no longer make sense, and there will be a need for a broader societal conversation about how the economy should be organized.
然而，我确实认为从长远来看，人工智能将变得如此广泛有效且成本低廉，以至于这一情况将不再适用。到那时，我们当前的经济结构将不再合理，社会需要进行更广泛的讨论，探讨经济应如何组织。

While that might sound crazy, the fact is that civilization has successfully navigated major economic shifts in the past: from hunter-gathering to farming, farming to feudalism, and feudalism to industrialism. I suspect that some new and stranger thing will be needed, and that it’s something no one today has done a good job of envisioning. It could be as simple as a large universal basic income for everyone, although I suspect that will only be a small part of a solution. It could be a capitalist economy of AI systems, which then give out resources (huge amounts of them, since the overall economic pie will be gigantic) to humans based on some secondary economy of what the AI systems think makes sense to reward in humans (based on some judgment ultimately derived from human values). Perhaps the economy runs on Whuffie points. Or perhaps humans will continue to be economically valuable after all, in some way not anticipated by the usual economic models. All of these solutions have tons of possible problems, and it’s not possible to know whether they will make sense without lots of iteration and experimentation. And as with some of the other challenges, we will likely have to fight to get a good outcome here: exploitative or dystopian directions are clearly also possible and have to be prevented. Much more could be written about these questions and I hope to do so at some later time.
虽然这听起来可能很疯狂，但事实是文明已经成功应对了过去重大的经济转型：从狩猎采集到农耕，从农耕到封建主义，从封建主义到工业主义。我猜想未来需要某种更新奇的事物，而这是当今任何人都未能清晰构想的。它可能简单到只是为所有人提供大范围的全民基本收入，尽管我怀疑这只会是解决方案的一小部分。也可能是由人工智能系统运行的资本主义经济，然后根据某种次级经济体系（基于最终源自人类价值观的某种判断，由 AI 系统决定对人类值得奖励的行为）向人类分配资源（资源量将极其庞大，因为整体经济规模会变得巨大）。或许经济体系会以 Whuffie 点数运行。又或许人类终将以某种常规经济模型无法预见的方式继续保持经济价值。所有这些解决方案都存在大量潜在问题，不经过大量迭代和实验，我们无法判断它们是否合理。 与其他一些挑战一样，我们很可能需要努力争取才能获得好的结果：剥削性或反乌托邦的方向显然也是可能的，必须加以防范。关于这些问题还有很多可以讨论的内容，我希望在以后的时间里继续探讨。

Taking stock 盘点现状

Through the varied topics above, I’ve tried to lay out a vision of a world that is both plausible if everything goes right with AI, and much better than the world today. I don’t know if this world is realistic, and even if it is, it will not be achieved without a huge amount of effort and struggle by many brave and dedicated people. Everyone (including AI companies!) will need to do their part both to prevent risks and to fully realize the benefits.
通过上述多元议题，我试图勾勒出一个愿景：若人工智能发展一切顺利，这个世界不仅切实可行，还将远胜于今日。虽不知此愿景是否现实，但即便可行，也需无数勇敢坚定者付出巨大努力与抗争方能实现。所有人（包括 AI 公司！）都需各尽其责，既要防范风险，也要充分释放技术红利。

But it is a world worth fighting for. If all of this really does happen over 5 to 10 years—the defeat of most diseases, the growth in biological and cognitive freedom, the lifting of billions of people out of poverty to share in the new technologies, a renaissance of liberal democracy and human rights—I suspect everyone watching it will be surprised by the effect it has on them. I don’t mean the experience of personally benefiting from all the new technologies, although that will certainly be amazing. I mean the experience of watching a long-held set of ideals materialize in front of us all at once. I think many will be literally moved to tears by it.
但这正是值得奋斗的未来。倘若未来 5 到 10 年真能见证这些变革——攻克多数疾病、拓展生物与认知自由、数十亿人脱贫共享新技术、自由主义民主与人权复兴——我猜想每位见证者都将被其震撼。这不仅指亲身受益于新科技的奇妙体验（尽管那必然令人惊叹），更是目睹长期秉持的理想骤然成真的集体震撼。届时，许多人定会感动落泪。

Throughout writing this essay I noticed an interesting tension. In one sense the vision laid out here is extremely radical: it is not what almost anyone expects to happen in the next decade, and will likely strike many as an absurd fantasy. Some may not even consider it desirable; it embodies values and political choices that not everyone will agree with. But at the same time there is something blindingly obvious—something overdetermined—about it, as if many different attempts to envision a good world inevitably lead roughly here.
在撰写这篇文章的过程中，我注意到一种有趣的矛盾。从某种意义上说，这里提出的愿景极为激进：它几乎不是任何人预期在未来十年内会发生的事情，而且很可能会让许多人觉得这是一个荒谬的幻想。有些人甚至可能认为它并不可取；它体现的价值观和政治选择并非所有人都能认同。但与此同时，这其中又有某种显而易见——甚至可以说是过度确定——的东西，仿佛许多试图构想美好世界的尝试都不可避免地大致指向这里。

In Iain M. Banks’ The Player of Games, the protagonist—a member of a society called the Culture, which is based on principles not unlike those I’ve laid out here—travels to a repressive, militaristic empire in which leadership is determined by competition in an intricate battle game. The game, however, is complex enough that a player’s strategy within it tends to reflect their own political and philosophical outlook. The protagonist manages to defeat the emperor in the game, showing that his values (the Culture’s values) represent a winning strategy even in a game designed by a society based on ruthless competition and survival of the fittest. A well-known post by Scott Alexander has the same thesis—that competition is self-defeating and tends to lead to a society based on compassion and cooperation. The “ arc of the moral universe ” is another similar concept.
在伊恩·M·班克斯的《游戏玩家》 中，主角——一个名为"文明"社会的成员，该社会的原则与我在此阐述的理念颇为相似——前往一个压迫性的军国主义帝国，那里的领导权由一场复杂战争游戏的竞争决定。然而，这种游戏的精妙之处在于，玩家的策略往往反映出其政治与哲学观。主角最终在游戏中击败了皇帝，证明了他的价值观（即"文明"的价值观）即便在这个基于残酷竞争与适者生存的社会设计的游戏中，也能成为制胜策略。斯科特·亚历山大的一篇著名文章提出了相同论点——竞争会自我瓦解，并终将导向以同情与合作基础的社会。"道德宇宙的弧线"则是另一个相似概念。

I think the Culture’s values are a winning strategy because they’re the sum of a million small decisions that have clear moral force and that tend to pull everyone together onto the same side. Basic human intuitions of fairness, cooperation, curiosity, and autonomy are hard to argue with, and are cumulative in a way that our more destructive impulses often aren’t. It is easy to argue that children shouldn’t die of disease if we can prevent it, and easy from there to argue that everyone’s children deserve that right equally. From there it is not hard to argue that we should all band together and apply our intellects to achieve this outcome. Few disagree that people should be punished for attacking or hurting others unnecessarily, and from there it’s not much of a leap to the idea that punishments should be consistent and systematic across people. It is similarly intuitive that people should have autonomy and responsibility over their own lives and choices. These simple intuitions, if taken to their logical conclusion, lead eventually to rule of law, democracy, and Enlightenment values. If not inevitably, then at least as a statistical tendency, this is where humanity was already headed. AI simply offers an opportunity to get us there more quickly—to make the logic starker and the destination clearer.
我认为《文明》的价值观之所以是制胜策略，在于它们源自无数细微抉择的集合——这些抉择既蕴含明确的道德力量，又能将所有人凝聚到同一阵营。人类与生俱来的公平、合作、好奇与自主等直觉难以辩驳，且具有累积效应，这与我们更具破坏性的冲动形成鲜明对比。若可预防却放任儿童死于疾病，这种观点显然站不住脚；由此自然推导出所有儿童都应平等享有生存权。进而我们不难达成共识：应当集结众人智慧来实现这一目标。几乎无人反对对无端伤害他人者施以惩戒，由此稍加延伸便能形成"惩罚应跨人群保持连贯与系统性"的观念。同样符合直觉的是：每个人都应对自身生活与选择享有自主权并承担责任。这些朴素直觉若推演至逻辑终点，终将导向法治、民主与启蒙运动的价值观。 即使不是必然，至少作为一种统计趋势，这正是人类已经行进的方向。人工智能只是提供了一个让我们更快抵达那里的机会——使逻辑更加鲜明，目的地更加清晰。

Nevertheless, it is a thing of transcendent beauty. We have the opportunity to play some small role in making it real.
然而，这是一件具有超凡美感的事物。我们有机会在实现它的过程中扮演微小的角色。

Thanks to Kevin Esvelt, Parag Mallick, Stuart Ritchie, Matt Yglesias, Erik Brynjolfsson, Jim McClave, Allan Dafoe, and many people at Anthropic for reviewing drafts of this essay.
感谢 Kevin Esvelt、Parag Mallick、Stuart Ritchie、Matt Yglesias、Erik Brynjolfsson、Jim McClave、Allan Dafoe 以及 Anthropic 的许多人对本文草稿的审阅。

To the winners of the 2024 Nobel prize in Chemistry, for showing us all the way.
致 2024 年诺贝尔化学奖得主，感谢你们为我们指明方向。

Footnotes 脚注

  1. 1 https://allpoetry.com/All-Watched-Over-By-Machines-Of-Loving-Grace
  2. 2 I do anticipate some minority of people’s reaction will be “this is pretty tame”. I think those people need to, in Twitter parlance, “touch grass”. But more importantly, tame is good from a societal perspective. I think there’s only so much change people can handle at once, and the pace I’m describing is probably close to the limits of what society can absorb without extreme turbulence.
    2 我确实预见到少数人的反应会是“这相当温和”。我认为这些人需要，用推特上的话说，“接接地气”。但更重要的是，从社会角度来看，温和是好事。我觉得人们一次性能承受的变化有限，而我描述的速度可能已接近社会在不引发剧烈动荡的情况下所能消化的极限。↩
  3. 3 I find AGI to be an imprecise term that has gathered a lot of sci-fi baggage and hype. I prefer "powerful AI" or "Expert-Level Science and Engineering" which get at what I mean without the hype.
    3 我认为 AGI 是个不够精确的术语，附带了太多科幻色彩和炒作。我更喜欢用“强大 AI”或“专家级科学与工程”这类表述，它们能更准确地传达我的意思而不带夸张。↩
  4. 4 In this essay, I use "intelligence" to refer to a general problem-solving capability that can be applied across diverse domains. This includes abilities like reasoning, learning, planning, and creativity. While I use "intelligence" as a shorthand throughout this essay, I acknowledge that the nature of intelligence is a complex and debated topic in cognitive science and AI research. Some researchers argue that intelligence isn't a single, unified concept but rather a collection of separate cognitive abilities. Others contend that there's a general factor of intelligence (g factor) underlying various cognitive skills. That’s a debate for another time.
    4 在本文中，我用“智能”来指代一种可应用于不同领域的通用问题解决能力。这包括推理、学习、规划和创造力等能力。虽然我在全文中用“智能”作为简称，但我承认智能的本质是认知科学和人工智能研究中一个复杂且有争议的话题。一些研究者认为智能并非单一、统一的概念，而是一系列独立的认知能力集合。另一些人则认为存在一个支撑各种认知技能的通用智力因素（g 因素）。这个争论留待日后探讨。↩
  5. 5 This is roughly the current speed of AI systems – for example they can read a page of text in a couple seconds and write a page of text in maybe 20 seconds, which is 10-100x the speed at which humans can do these things. Over time larger models tend to make this slower but more powerful chips tend to make it faster; to date the two effects have roughly canceled out.
    5 这大致是当前 AI 系统的速度——例如，它们能在几秒内读完一页文本，并在大约 20 秒内写出一页文本，速度是人类完成相同任务的 10 到 100 倍。随着时间的推移，更大的模型往往会使其变慢，但更强大的芯片往往会使其加快；迄今为止，这两种效应大致相互抵消。↩
  6. 6 This might seem like a strawman position, but careful thinkers like Tyler Cowen and Matt Yglesias have raised it as a serious concern (though I don’t think they fully hold the view), and I don’t think it is crazy.
    6 这看似是个稻草人论点，但像泰勒·考恩和马特·伊格莱西亚斯这样严谨的思考者都将其视为一个严肃问题（尽管我不认为他们完全持此观点），而且我并不觉得这种担忧是荒谬的。↩
  7. 7 The closest economics work that I’m aware of to tackling this question is work on “general purpose technologies” and “ intangible investments ” that serve as complements to general purpose technologies.
    7 据我所知，经济学领域最接近解决这个问题的研究是关于‘通用目的技术’及其补充‘无形资产投资’的探讨。↩
  8. 8 This learning can include temporary, in-context learning, or traditional training; both will be rate-limited by the physical world.
    8 这种学习可以包括临时的、上下文相关的学习，或是传统训练；两者都将受到物理世界的速率限制。↩
  9. 9 In a chaotic system, small errors compound exponentially over time, so that even an enormous increase in computing power leads to only a small improvement in how far ahead it is possible to predict, and in practice measurement error may degrade this further.
    9 在混沌系统中，微小误差会随时间呈指数级累积，因此即使计算能力大幅提升，也只能略微延长可预测的时间范围，而实际测量误差可能进一步削弱这种改进。↩
  10. 10 Another factor is of course that powerful AI itself can potentially be used to create even more powerful AI. My assumption is that this might (in fact, probably will) occur, but that its effect will be smaller than you might imagine, precisely because of the “decreasing marginal returns to intelligence” discussed here. In other words, AI will continue to get smarter quickly, but its effect will eventually be limited by non-intelligence factors, and analyzing those is what matters most to the speed of scientific progress outside AI.
    10 当然，另一个因素是强大的 AI 本身可能被用来创造更强大的 AI。我的假设是这种情况可能会（事实上很可能将会）发生，但其影响会比想象中要小，正是因为这里讨论的“智力边际收益递减”。换句话说，AI 将继续快速变得更聪明，但其影响最终会受到非智力因素的限制，而分析这些因素对 AI 之外的科学进步速度最为重要。↩
  11. 11 These achievements have been an inspiration to me and perhaps the most powerful existing example of AI being used to transform biology.
    11 这些成就对我是一种激励，或许是现有最有力的例证，展示了 AI 如何被用于变革生物学领域。↩
  12. 12 “Progress in science depends on new techniques, new discoveries and new ideas, probably in that order.” - Sydney Brenner
    12 “科学进步依赖于新技术、新发现和新思想，很可能按此顺序。” ——悉尼·布伦纳↩
  13. 13 Thanks to Parag Mallick for suggesting this point.
    13 感谢 Parag Mallick 提出这一点。↩
  14. 14 I didn't want to clog up the text with speculation about what specific future discoveries AI-enabled science could make, but here is a brainstorm of some possibilities:
    14 我不想在文中堆砌对 AI 赋能科学未来可能具体发现的各种猜测，但这里列举一些可能性供头脑风暴：
    — Design of better computational tools like AlphaFold and AlphaProteo — that is, a general AI system speeding up our ability to make specialized AI computational biology tools.
    ——设计更优的计算工具，如 AlphaFold 和 AlphaProteo——即通过通用 AI 系统加速我们开发专用 AI 计算生物学工具的能力。
    — More efficient and selective CRISPR.
    ——更高效、更具选择性的 CRISPR 技术。
    — More advanced cell therapies.
    ——更先进的细胞疗法。
    — Materials science and miniaturization breakthroughs leading to better implanted devices.
    材料科学与微型化技术的突破，推动植入式设备性能提升。
    — Better control over stem cells, cell differentiation, and de-differentiation, and a resulting ability to regrow or reshape tissue.
    增强对干细胞、细胞分化及去分化过程的调控能力，进而实现组织再生或重塑。
    — Better control over the immune system: turning it on selectively to address cancer and infectious disease, and turning it off selectively to address autoimmune diseases.
    提升免疫系统精准调控水平：选择性激活以应对癌症和传染病，选择性抑制以治疗自身免疫疾病。↩
  15. 15 AI may of course also help with being smarter about choosing what experiments to run: improving experimental design, learning more from a first round of experiments so that the second round can narrow in on key questions, and so on.
    15 人工智能当然也能助力更明智地选择实验方向：优化实验设计，从首轮实验中汲取更多信息，使后续实验能聚焦关键问题，诸如此类。↩
  16. 16 Thanks to Matthew Yglesias for suggesting this point.
    16 感谢马修·伊格莱西亚斯提出这一点。↩
  17. 17 Fast evolving diseases, like the multidrug resistant strains that essentially use hospitals as an evolutionary laboratory to continually improve their resistance to treatment, could be especially stubborn to deal with, and could be the kind of thing that prevents us from getting to 100%.
    17 快速进化的疾病，比如那些多重耐药菌株，它们基本上将医院作为进化实验室，持续增强对治疗的抵抗力，可能会特别难以应对，并可能成为阻碍我们达到 100%治愈率的因素。↩
  18. 18 Note it may be hard to know that we have doubled the human lifespan within the 5-10 years. While we might have accomplished it, we may not know it yet within the study time-frame.
    18 需要注意的是，在 5-10 年内实现人类寿命翻倍这一成就可能难以被即时察觉。即便我们已经达成这一目标，在研究的时间框架内或许还无法确知。↩
  19. 19 This is one place where I am willing, despite the obvious biological differences between curing diseases and slowing down the aging process itself, to instead look from a greater distance at the statistical trend and say “even though the details are different, I think human science would probably find a way to continue this trend; after all, smooth trends in anything complex are necessarily made by adding up very heterogeneous components.
    19 此处，尽管治愈疾病与延缓衰老过程之间存在显著的生物学差异，我仍愿意从更宏观的统计趋势视角出发，提出这样的观点：“即便具体细节各异，我认为人类科学很可能找到延续这一趋势的方法；毕竟，任何复杂事物中平滑的趋势必然是由高度异质的组成部分叠加而成。”↩
  20. 20 As an example, I’m told that an increase in productivity growth per year of 1% or even 0.5% would be transformative in projections related to these programs. If the ideas contemplated in this essay come to pass, productivity gains could be much larger than this.
    20 举个例子，有人告诉我，每年生产率增长提高 1%甚至 0.5%，就会对这些项目的相关预测产生变革性影响。如果本文所设想的想法得以实现，生产率的提升可能会远大于此。↩
  21. 21 The media loves to portray high status psychopaths, but the average psychopath is probably a person with poor economic prospects and poor impulse control who ends up spending significant time in prison.
    21 媒体热衷于描绘高地位的精神病患者，但普通的精神病患者很可能是一个经济前景不佳、冲动控制能力差的人，最终会在监狱里度过大量时间。↩
  22. 22 I think this is somewhat analogous to the fact that many, though likely not all, of the results we’re learning from interpretability would continue to be relevant even if some of the architectural details of our current artificial neural nets, such as the attention mechanism, were changed or replaced in some way.
    22 我认为这与我们从可解释性中学到的许多结果（尽管可能不是全部）在某种程度上是类似的，即使我们当前人工神经网络的一些架构细节（如注意力机制）以某种方式被改变或替换，这些结果仍将继续相关。↩
  23. 23 I suspect it is a bit like a classical chaotic system – beset by irreducible complexity that has to be managed in a mostly decentralized manner. Though as I say later in this section, more modest interventions may be possible. A counterargument, made to me by economist Erik Brynjolfsson, is that large companies (such as Walmart or Uber) are starting to have enough centralized knowledge to understand consumers better than any decentralized process could, perhaps forcing us to revise Hayek’s insights about who has the best local knowledge.
    23 我怀疑这有点像经典的混沌系统——受到不可约复杂性的困扰，必须以一种基本分散的方式进行管理。不过正如我在本节后面提到的，更温和的干预措施或许是可能的。经济学家埃里克·布林约尔松向我提出的一个反论点是，大公司（如沃尔玛或优步）正开始拥有足够的集中化知识，能够比任何分散化过程更好地理解消费者，这可能迫使我们重新审视哈耶克关于谁拥有最佳本地知识的见解。↩
  24. 24 Thanks to Kevin Esvelt for suggesting this point.
    24 感谢凯文·埃斯维尔特提出这一点。↩
  25. 25 For example, cell phones were initially a technology for the rich, but quickly became very cheap with year-over-year improvements happening so fast as to obviate any advantage of buying a “luxury” cell phone, and today most people have phones of similar quality.
    例如，手机最初是面向富人的技术，但随着逐年快速改进，很快变得非常便宜，以至于购买“奢侈”手机的优势不复存在，如今大多数人拥有的手机质量相近。↩
  26. 26 This is the title of a forthcoming paper from RAND, that lays out roughly the strategy I describe.
    这是兰德公司即将发表的一篇论文标题，大致描述了我所阐述的策略。↩
  27. 27 When the average person thinks of public institutions, they probably think of their experience with the DMV, IRS, medicare, or similar functions. Making these experiences more positive than they currently are seems like a powerful way to combat undue cynicism.
    当普通人想到公共机构时，他们可能会联想到在车管所、国税局、医疗保险或类似部门的经历。改善这些现状中的体验，似乎是消除过度怀疑态度的有效方式。↩
  28. 28 Indeed, in an AI-powered world, the range of such possible challenges and projects will be much vaster than it is today.
    确实，在人工智能驱动的世界里，这类可能的挑战和项目的范围将比今天广阔得多。↩
  29. 29 I am breaking my own rule not to make this about science fiction, but I’ve found it hard not to refer to it at least a bit. The truth is that science fiction is one of our only sources of expansive thought experiments about the future; I think it says something bad that it’s entangled so heavily with a particular narrow subculture.
    29 我打破了自己不谈科幻的规矩，但发现很难完全不提及它。事实上，科幻是我们仅有的几个对未来进行广阔思想实验的来源之一；我认为它与某个狭隘亚文化如此紧密纠缠的事实，本身就说明了某种问题。↩